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DETAILED ACTION 
Claim Rejections - 35 USC § 112 

The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

Claims 15-20 are rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. In particular, there is no antecedent basis for "the graphic user interface," which is 
recited in the claim. As claims 16-20 depend on claim 15 and include all of the limitations of 
claim 15, claims 16-20 are similarly considered indefinite. 



Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1-31 are rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. Patent 
No. 6,438,523, which is attributed to OberteufFer et al. (hereafter referred to as "Oberteuffer"), 
and also over U.S. Patent No. 5,914,707, which is attributed to Kono. In general, Oberteuffer 
presents a computer system which is capable of processing both speech input and hand-drawn 
input (see column 3, lines 10-28). This computer system responds to such inputs by graphically 
displaying text or graphics accordingly (for example, see column 5, lines 24-45). Lastly, it is 
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interpreted that such a computer system may be implemented as a book-sized computer, i.e. an 
"electronic book device" (see column 2, lines 51-63). 

Thus regarding claims 1, 2, and 4, Oberteuffer presents an electronic book device which 
comprises a portable housing and a processor, as is known in the art, and whereby the processor 
displays content stored in a storage device by responding to plural input modes. Specifically 
regarding claim 4, these input modes include a speech input mode and a hand-drawn input mode, 
or in other words, a sound input mode and a graphics input mode. The electronic book device of 
Oberteuffer subsequently comprises a "speech interface" and a "pen interface/' whereby this 
speech interface includes an audio input device, specifically a microphone, and this pen interface 
includes a graphics input device, specifically an electronic pen (see column 4, lines 1-16). 
Therefore, it is understood that the electronic book of Oberteuffer implements at least a graphics 
input mode and a sound input mode, whereby the electronic book includes at least one graphics 
input device and at least one audio input device, both sending input signals to the processor. It is 
further interpreted that the electronic book device of Oberteuffer implements a visual graphics 
output mode, whereby the electronic book comprises a visual display, which is responsive to the 
processor for outputting content (for example, see column 6, lines 2-23). However, Oberteuffer 
does not explicitly disclose that the electronic book is capable of outputting content using plural 
output modes, as is expressed in claim 1. Oberteuffer consequently does not disclose that these 
plural output modes include at least visual graphics and sound, and that the electronic book 
includes a visual display and at least one audio speaker, both being responsive to the processor 
for outputting content, as is expressed in claim 2. 
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Like Oberteuffer, Kono discusses electronic books capable of inputting data graphically 
and orally, wherein these electronic books respond to such inputs by outputting text or graphics 
accordingly (for example, see column 1, lines 13-52). Moreover, and specifically regarding the 
claimed invention, Kono discloses that such an electronic book may output audio data as well as 
this graphical data (see column 3, lines 39-45). For example, audio speech may accompany 
displayed text (see column 7, lines 29-32). In order to do so, the electronic book comprises a 
speaker in order to output audio data (see column 5, lines 7-23). Kono thus presents an 
electronic book which responds to input and is capable of outputting content using plural output 
modes, the plural output modes including at least visual graphics and sound, and wherein the 
device includes at least one visual display and at least one audio speaker, both being responsive 
to a processor for outputting content. 

It would have therefore been obvious to one of ordinary skill in the art, having the 
teachings of Oberteuffer and Kono before him at the time the invention was made, to modify the 
electronic book device taught by Oberteuffer such that it also outputs content via an audio output 
mode, as is done by the electronic book device of Kono. It would have been advantageous to 
one of ordinary skill to utilize such a combination because audio output further enhances 
understanding of graphically displayed content, as is demonstrated by Kono. 

As per claim 3, Kono teaches that the electronic book is responsive to user input selecting 
an audio output mode. For example, Kono discloses that the electronic book outputs audio 
speech in response to the user selecting a particular icon (see column 7, lines 29-32). It is 
therefore understood that with the above-described combination of Oberteuffer and Kono, the 
processor of the electronic book is responsive to user input selecting an output mode. 
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In reference to claim 5, Oberteuffer teaches that the electronic book is responsive to user 
input selecting an input mode (see column 7, lines 10-24). It is therefore understood that with 
the above-described combination of Oberteuffer and Kono, the processor of the electronic book 
is responsive to user input selecting an input mode. 

Concerning claim 6, Oberteuffer discloses that the electronic book responds to a graphics 
input mode by outputting content using a graphic user interface. For example, in response to a 
user inputting a hand-drawn image, the processor of the electronic book responds by displaying a 
corresponding "clean" representation of the image (see column 4, lines 45-53). It is therefore 
understood that with the above-described combination of Oberteuffer and Kono, the processor of 
the electronic book responds to a graphics input mode my outputting content in a graphics output 
mode using a graphic user interface. Continuing further with claim 6, Kono discloses that audio 
information is output in response to the selection of a particular "voice" icon (see column 7, lines 
29-32). It is interpreted that with the electronic book of Oberteuffer and Kono, speech 
commands may be used to select such icons (for example, see column 4, lines 54-61 of 
Oberteuffer). Thus with a speech command being used to select a voice icon, the processor 
responds to an audio input mode by outputting content in an audio output mode using an audio 
user interface. 

In regard to claim 13, the processor of the above-described electronic book of Oberteuffer 
and Kono is considered an "abstract interface," like that of the claimed invention, wherein this 
processor is responsible for accessing content stored in the data storage of the electronic book as 
is known in the art. In addition, Oberteuffer discloses that the electronic book may receive 
commands via its audio user interface, whereby it updates its graphical user interface in response 
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thereto (for example, see column 5, lines 57-65). Kono similarly teaches that the electronic book 
may receive commands via its graphical user interface, whereby it updates its audio user 
interface in response thereto. Specifically, it is understood that commands to change a 
graphically displayed page of a document may be entered via the graphical user interface, 
whereby the speech interface is updated in order to reflect the currently displayed page (see 
column 7, lines 29-39). It is therefore understood that the above-described electronic book taught 
by Oberteuffer and Kono comprises: content stored in a data storage; an abstract interface 
accessing the data storage; an audio user interface communicating with the abstract interface; 
and a graphics user interface communicating with the abstract interface, the abstract interface 
receiving user input commands from the audio user interface and updating the graphics user 
interface in response thereto, the abstract user interface receiving user input commands from the 
graphics user interface and updating the audio user interface in response thereto. 

In reference to claims 14 and 15, the electronic book of Oberteuffer and Kono includes a 
visual display associated with its graphics user interface and an audio speaker associated with its 
audio user interface, as is shown above in the rejection for claims 1, 2, and 4. Specifically, as is 
shown above, content is output in a graphics output mode using the graphics user interface, and 
content is output in an audio output mode using the audio user interface. 

Regarding claims 7 and 16, Kono discloses that an electronic book may output both audio 
data and video data concurrently (see column 3, lines 39-45). Similarly, Oberteuffer discloses 
that a user may enter data into an electronic book by simultaneous graphical and oral commands 
(for example, see column 4, linesl7-64; and column 5, lines 24-45). It is therefore understood 
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that the audio user interface and the graphical user interface of above-described electronic book 
of Oberteuffer and Kono may run simultaneously with each other. 

As per claims 8, 10, 11, 17, and 19, Oberteuffer discloses that the electronic book may 
receive annotations from a user-selected interface, wherein the annotations are associated with 
user-selected portions of content, and interpreted to be stored by the electronic book. Moreover, 
the other user interface is updated with the annotations. Oberteuffer specifically discloses that a 
user may enter annotations via a speech interface (see column 5, lines 24-45), which may be 
user-selected (see column 7, lines 1 1-24), and whereby the annotations are displayed via the 
graphical user interface and are associated with an "organizational structure" displayed by the 
graphical user interface (see column 5, lines 24-45). It is therefore understood that with the 
above-described combination of Oberteuffer and Kono, the electronic book's processor receives 
for storage annotations from a user-selected user interface, wherein the annotations are 
associated with user-selected portions of content, and wherein the processor updates another user 
interface with the annotations. 

Concerning claims 9, 12, 18, and 20, Kono teaches that a user may navigate through 
content via a user-selected user interface of the electronic book, wherein the other user interface 
is updated to reflect such navigation. For example, Kono discloses that a user may visually 
navigate, i.e. read, a displayed document, and wherein a voice icon associated with the current 
page may be selected in order to provide audio accompaniment of the displayed page (see 
column 7, lines 29-39). Thus a user may navigate through content via a graphical user interface, 
wherein the audio user interface is updated to reflect the currently displayed page. It is therefore 
understood that with the above-described combination of Oberteuffer and Kono, the electronic 
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book's processor is programmed to allow a user to navigate through content using a user-selected 
user interface to render a navigation result, whereby the processor updates another user interface 
with the navigation result. 

With respect to claim 21, the above-described electronic book of Oberteuffer and Kono is 
considered a "computer program product" like that of the claimed invention, as it is understood 
that such an electronic book comprises a computer program storage device and computer- 
readable instructions on the storage device, as is known in the art (see also, column 8, line 46 - 
column 10, line 26 of Kono). Moreover, and for the reasons described above in the rejection for 
claims 1, 2, and 4, it is understood that these computer-readable instructions cause the electronic 
book to display electronic content in more than one mode. It is further understood that such 
instructions include computer-readable code means for receiving an annotation to content via an 
audio user interface, associating the annotation with the content, and displaying the annotation 
and the associated content using a graphical user interface, as is shown above in the rejection for 
claims 8, 10, 11, 17, and 19. 

Referring to claim 22, Oberteuffer teaches that a user may add text to a memo using a 
graphical user interface (for example, see column 5, lines 58-65). It is therefore understood that 
the user may add annotations to the memo using the graphical user interface, whereby it is 
understood that the annotations are associated with the memo. Moreover, Kono teaches that a 
document, like this memo, may be reproduced using an audio user interface (see column 7, lines 
29-39). It is therefore understood that the above-described electronic book of Oberteuffer and 
Kono comprises: computer-readable code means for receiving an annotation to content via a 
graphical user interface; computer-readable code means for associating the annotation with the 
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content; and computer-readable code means for displaying the annotation and associated content 
using an audio user interface. 

Concerning claim 23 , the above-described audio user interface and graphical user 
interface of the electronic book of Oberteuffer and Kono run simultaneously with each other, as 
is described above in the rejection for claims 7 and 16. 

As per claims 24 and 26, the above-described electronic book of Oberteuffer and Kono is 
understood to comprise computer readable code means for allowing a user to navigate through 
the content using a user selected user interface in order to render a navigation result, whereby 
another user interface is updated with the navigation result, as is described above in the rejection 
for claims 9, 12, 18, and 20. 

Regarding claim 25, the above-described electronic book of Oberteuffer and Kono is 
understood to comprise computer-readable code means for storing annotations from a user- 
selected user interface, whereby another user interface is updated with the annotations, as is 
described above in the rejection for claims 8, 10, 11, 17 and 19. 

In regard to claims 27 and 28, the above-described electronic book of Oberteuffer and 
Kono is understood to teach a method for presenting content using an electronic book. In 
particular, and for the reasons described above in the rejection for claims 1, 2, and 4, it is 
understood that such a method comprises providing a portable housing having content 
electrically stored therein, and simultaneously implementing two modes of output for displaying 
the content. It is understood that these modes of output, specifically a graphical user interface 
and audio user interface, are each implemented by a "thread," as is known in the art. 
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Concerning claim 29, the above-described audio user interface and graphical user 
interface threads of the electronic book of Oberteuffer and Kono run simultaneously with each 
other, as is described above in the rejection for claims 7 and 16. Moreover, it is interpreted that 
these threads are at the same location in the content (for example, see column 7, lines 29-39 of 
Kono; the speech output thread and audio output thread are at the same page of a document). 

In regard to claims 30 and 3 1 , it is understood that each output thread is associated with a 
corresponding input mode, either a graphical input mode or an audio input mode, for inputting 
annotations and user commands, as is described above in the rejection for claim 6. As 
Oberteuffer teaches that such input modes may be switched in response to user commands (see 
column 7, lines 1 1-24), it is interpreted that the above-described electronic book of Oberteuffer 
and Kono is similarly responsive to user commands to switch from displaying content from 
displaying the content using the first output thread to displaying the content using the second 
output thread. 

Conclusion 

The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. The applicant is required under 37 C.F.R. §1.11 1(C) to 
consider these references fully when responding to this action. The Ozawa et al. U.S. Patent 
cited therein presents device which comprises a graphical user interface and a speech interface 
for receiving graphical commands and speech commands, and which is for outputting graphics 
and sound. The Ando et al. U.S. Patent cited therein presents a system which comprises a speech 
interface and a graphical user interface. Lastly, the Diel et al. U.S. Patent cited therein presents a 
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system which comprises different user interface types, and which allows the user to switch 
between these interface types. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Blaine Basom whose telephone number is (703) 305-7694. The 
examiner can normally be reached on Monday through Friday, from 8:30 am to 5:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on (703) 308-3 116. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 305-3900. 
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